神经辐射场（NERF）为3D互动体验带来了新的浪潮。但是，作为沉浸式体验的重要组成部分，在NERF中尚未完全探索散焦效应。最近一些基于NERF的方法通过利用多平台技术以后处理方式产生3D散焦效果。尽管如此，它们还是耗时的，或者是记忆力消费。本文提出了一种新型的基于薄镜的NERF框架，该框架可以直接呈现出各种3D散焦效应，称为nerfocus。与针孔不同，薄镜头折射了场景点的光线，因此其在传感器平面上的成像被散布为混乱圆（COC）。直接的解决方案采样足够的射线以近似此过程在计算上很昂贵。取而代之的是，我们建议将薄镜头成像倒数，以明确对传感器平面上每个点的光束路径进行建模，并将此范式推广到每个像素的梁路径，然后使用基于flustum的体积渲染以渲染每个像素的梁路径。我们进一步设计了有效的概率培训（P-Training）策略，以大大简化培训过程。广泛的实验表明，我们的nerfocus可以通过可调节的相机姿势，聚焦距离和光圈大小实现各种3D散焦效应。通过将孔径大小设置为零，可以将现有的NERF视为我们的特殊情况。尽管有这样的优点，但Nerfocus并未牺牲Nerf的原始表现（例如培训和推理时间，参数消耗，渲染质量），这意味着其具有更广泛应用和进一步改进的巨大潜力。代码和视频可在https://github.com/wyhuai/nerfocus上找到。

尽管神经形态计算的快速进展，但尖刺神经网络（SNNS）的能力不足和不足的表现力严重限制了其在实践中的应用范围。剩余学习和捷径被证明是培训深层神经网络的重要方法，但以前的工作评估了他们对基于尖峰的通信和时空动力学的特征的适用性。在本文中，我们首先确定这种疏忽导致受阻信息流程和伴随以前的残留SNN中的降解问题。然后，我们提出了一种新型的SNN定向的残余块MS-Reset，能够显着地扩展直接训练的SNN的深度，例如，在ImageNet上最多可在CiFar-10和104层上完成482层，而不会观察到任何轻微的降级问题。我们验证了基于帧和神经形态数据集的MS-Reset的有效性，并且MS-Resnet104在直接训练的SNN的域中的第一次实现了在ImageNet上的76.02％精度的优越结果。还观察到巨大的能量效率，平均仅需要每根神经元的一穗来分类输入样本。我们相信我们强大且可扩展的型号将为进一步探索SNN提供强大的支持。

作为生成部件作为自回归模型的向量量化变形式自动化器（VQ-VAE）的集成在图像生成上产生了高质量的结果。但是，自回归模型将严格遵循采样阶段的逐步扫描顺序。这导致现有的VQ系列模型几乎不会逃避缺乏全球信息的陷阱。连续域中的去噪扩散概率模型（DDPM）显示了捕获全局背景的能力，同时产生高质量图像。在离散状态空间中，一些作品已经证明了执行文本生成和低分辨率图像生成的可能性。我们认为，在VQ-VAE的富含内容的离散视觉码本的帮助下，离散扩散模型还可以利用全局上下文产生高保真图像，这补偿了沿像素空间的经典自回归模型的缺陷。同时，离散VAE与扩散模型的集成解决了传统的自回归模型的缺点是超大的，以及在生成图像时需要在采样过程中的过度时间的扩散模型。结果发现所生成的图像的质量严重依赖于离散的视觉码本。广泛的实验表明，所提出的矢量量化离散扩散模型（VQ-DDM）能够实现与低复杂性的顶层方法的相当性能。它还展示了在没有额外培训的图像修复任务方面与自回归模型量化的其他矢量突出的优势。

In person re-identification (ReID) tasks, many works explore the learning of part features to improve the performance over global image features. Existing methods extract part features in an explicit manner, by either using a hand-designed image division or keypoints obtained with external visual systems. In this work, we propose to learn Discriminative implicit Parts (DiPs) which are decoupled from explicit body parts. Therefore, DiPs can learn to extract any discriminative features that can benefit in distinguishing identities, which is beyond predefined body parts (such as accessories). Moreover, we propose a novel implicit position to give a geometric interpretation for each DiP. The implicit position can also serve as a learning signal to encourage DiPs to be more position-equivariant with the identity in the image. Lastly, a set of attributes and auxiliary losses are introduced to further improve the learning of DiPs. Extensive experiments show that the proposed method achieves state-of-the-art performance on multiple person ReID benchmarks.

Node classification for graph-structured data aims to classify nodes whose labels are unknown. While studies on static graphs are prevalent, few studies have focused on dynamic graph node classification. Node classification on dynamic graphs is challenging for two reasons. First, the model needs to capture both structural and temporal information, particularly on dynamic graphs with a long history and require large receptive fields. Second, model scalability becomes a significant concern as the size of the dynamic graph increases. To address these problems, we propose the Time Augmented Dynamic Graph Neural Network (TADGNN) framework. TADGNN consists of two modules: 1) a time augmentation module that captures the temporal evolution of nodes across time structurally, creating a time-augmented spatio-temporal graph, and 2) an information propagation module that learns the dynamic representations for each node across time using the constructed time-augmented graph. We perform node classification experiments on four dynamic graph benchmarks. Experimental results demonstrate that TADGNN framework outperforms several static and dynamic state-of-the-art (SOTA) GNN models while demonstrating superior scalability. We also conduct theoretical and empirical analyses to validate the efficiency of the proposed method. Our code is available at https://sites.google.com/view/tadgnn.

Background and Purpose: Colorectal cancer is a common fatal malignancy, the fourth most common cancer in men, and the third most common cancer in women worldwide. Timely detection of cancer in its early stages is essential for treating the disease. Currently, there is a lack of datasets for histopathological image segmentation of rectal cancer, which often hampers the assessment accuracy when computer technology is used to aid in diagnosis. Methods: This present study provided a new publicly available Enteroscope Biopsy Histopathological Hematoxylin and Eosin Image Dataset for Image Segmentation Tasks (EBHI-Seg). To demonstrate the validity and extensiveness of EBHI-Seg, the experimental results for EBHI-Seg are evaluated using classical machine learning methods and deep learning methods. Results: The experimental results showed that deep learning methods had a better image segmentation performance when utilizing EBHI-Seg. The maximum accuracy of the Dice evaluation metric for the classical machine learning method is 0.948, while the Dice evaluation metric for the deep learning method is 0.965. Conclusion: This publicly available dataset contained 5,170 images of six types of tumor differentiation stages and the corresponding ground truth images. The dataset can provide researchers with new segmentation algorithms for medical diagnosis of colorectal cancer, which can be used in the clinical setting to help doctors and patients.

This paper studies how to flexibly integrate reconstructed 3D models into practical 3D modeling pipelines such as 3D scene creation and rendering. Due to the technical difficulty, one can only obtain rough 3D models (R3DMs) for most real objects using existing 3D reconstruction techniques. As a result, physically-based rendering (PBR) would render low-quality images or videos for scenes that are constructed by R3DMs. One promising solution would be representing real-world objects as Neural Fields such as NeRFs, which are able to generate photo-realistic renderings of an object under desired viewpoints. However, a drawback is that the synthesized views through Neural Fields Rendering (NFR) cannot reflect the simulated lighting details on R3DMs in PBR pipelines, especially when object interactions in the 3D scene creation cause local shadows. To solve this dilemma, we propose a lighting transfer network (LighTNet) to bridge NFR and PBR, such that they can benefit from each other. LighTNet reasons about a simplified image composition model, remedies the uneven surface issue caused by R3DMs, and is empowered by several perceptual-motivated constraints and a new Lab angle loss which enhances the contrast between lighting strength and colors. Comparisons demonstrate that LighTNet is superior in synthesizing impressive lighting, and is promising in pushing NFR further in practical 3D modeling workflows. Project page: https://3d-front-future.github.io/LighTNet .

Despite the rapid progress of open-domain generation-based conversational agents, most deployed systems treat dialogue contexts as single-turns, while systems dealing with multi-turn contexts are less studied. There is a lack of a reliable metric for evaluating multi-turn modelling, as well as an effective solution for improving it. In this paper, we focus on an essential component of multi-turn generation-based conversational agents: context attention distribution, i.e. how systems distribute their attention on dialogue's context. For evaluation of this component, We introduce a novel attention-mechanism-based metric: DAS ratio. To improve performance on this component, we propose an optimization strategy that employs self-contained distractions. Our experiments on the Ubuntu chatlogs dataset show that models with comparable perplexity can be distinguished by their ability on context attention distribution. Our proposed optimization strategy improves both non-hierarchical and hierarchical models on the proposed metric by about 10% from baselines.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.